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PROVISIONAL SPECIFICATION 


Invention Title: REAL TIME TRANSLATOR 


The invention is described in the following statement: 


Real Time Translator 


This invention relates to a real time translator for providing multi language "spoken word" 
communication, conversation, and/or dialogue, conferencing and public address system. It is 
particularly related to a multilanguage conversation translator for the tourist, business or 
professional translation but is not limited to such use. 

Background of the Invention 

Arguably, the greatest ability the human race possesses is that of communication via 
sophisticated languages that have evolved over time. However, it is also the biggest barrier 
currently facing humankind. Even as the word "globalisation" is frequently used these days in 
the field of trade and business as well as many other areas of interaction between the different 
peoples of the world, the main "obstacle" to achieving tme globalisation are language barriers. 
This limits the ability to communicate & converse one-on-one between people who converse 
through one of the many different languages. 

• The tourist in a foreign country where he does not speak the language struggles to make 
himself understood for the most basic of requirements like asking for directions or making a 
purchase. 

• The businessperson at the end of a telephone line trying to make conversation with either 
a potential client or business colleague in another country when he does not speak the language. 

• The speaker wanting to address and communicate with an audience that speaks a different 
language in a conference or broadcast situation. 

Summarv of the invention 

The aim of the invention is to provide an electronic solution to the language barrier between 
languages for the spoken word. 

Broadly the invention provides a multilanguage conversation translator having dual voice 
paths operated by one or more sound cards and software so that conversation fi-om one person in 
one spoken word language is translated and received by a second person in a second spoken word 
language at the same time or substantially at the same time as conversation from the second 
person in the second spoken word language is translated and received by the first person whereby 
the two persons can undertake a normal conversation in normal time but in different spoken word 
languages. 
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The translator can be portable or hand-held or be an earpiece or the like. Other 
versions of the system can be attached to the telephone system or attached to a personal address 
system or the like. 

In accordance with the invention there is provided a real time translator comprising: 

(a) a voice receiver; 

(b) a voice to text converter; 

(c) a text-to-text spoken language converter for receiving a first language and 
translating to a second selected language; 

(d) a text to voice converter for converting the translated second selected language to 
a voice output; and 

(e) a voice emitter for emitting the voice output. 

In one form of the invention there is provided a real time translator comprising: 

(a) at least one voice receiver; 

(b) at least one voice to text converter; 

(c) at least one text to text spoken language converter for receiving a first selected 
language text and translating to a second selected language text and/or for receiving the second 
selected language text and translating to the first selected language text; 

(d) at least one text to voice converter for converting the translated first and/or second 
selected language to a voice output; and 

(e) at least one voice emitter for emitting the voice outputs. 

The real time translator could include two sound paths formed by two separate electronic 
sound manipulators with associated software such that the sound of the first voice in first 
language being received can be converted to text while the translated text into the second 
selected language is being converted to voice by the second separate electronic sound 
manipulator with associated software. The separate electronic sound manipulators may be two 
PC sound cards or the like, or two separate left and right channels of a single PC sound card or 
the like with separate software control. 

In a particular preferred form of the invention there is provided a portable real time translator 
comprising 


(a) first and second voice receivers for receiving first and second selected voice languages; 

(b) first and second voice to text converters; 

(c) at least one text to text spoken language converter for receiving a first selected 
language text and translating to a second selected langxiage text and/or for receiving the second 
selected language text and translating to the first selected language text; 

(d) first and second voice converters for converting the translated first and second 
selected language to first and second voice outputs; and 

(e) first and second voice emitters for emitting the voice outputs. 

There is an "overlap" in the processing of conversion of first and second voice conversions to 
or from text and/or with text to text voice language translation such that the lag time between 
receiving voice and emitting translated voice is within a reasonable conversation period. Such 
period can be of the order of 2 seconds. Further to simulate conversation the voice translation 
and emission is in voice phrases substantially corresponding with voice phrasing of input voice 
such that a continual flow of spaced voice phrases simulates conversations. Generally, such 
voice phrases are a sentence or part of a sentence. 

Still fiirther there may be such "overlap" in processing such that a first voice in a first 
language is received and translated and emitting translated voice simultaneously or apparently 
simultaneously with receiving a second voice in a second language and translating and emitting 
second translated voice. This can be by separate processing paths including the separate PC 
sound cards or the like or separate channels on a sound card or the like or by a switching system 
for switching between two processing paths at a rate to mamtain reasonable real time processing 
of both paths simultaneously. 

The invention also provides a method of providing real time translation of voices. The 
method includes: 

a) providing first and second voice receivers for receiving first and second selected 
voice languages; 

(b) providing first and second voice emitters associated with the first and second 
voice receivers respectively for emitting voice outputs; 

(c) converting said first and second selected voice languages from said first and 
second voice receivers to text; 


(d) providing a text to text spoken language converter for receiving a first selected 
language text fi-om said first voice receiver and translating to a second selected language text 
and/or for receiving the second selected language text and translating to the first selected 
language text; 

(e) providing a voice converter for converting the translated first and second selected 
language to first and second voice outputs; and 

(f) emitting said translated and converted first and second voice outputs. 

There is parallel processing of the voice to text conversion and/or text translation and/or the text to 
voice conversion. Two sound cards or two channels operating separately on a sound card can 
provide the first and second voice receivers and first and second voice emitters. Processing of the 
voice to text conversion and/or text translation and/or the text to voice conversion is by a cpu or the 
like with software control of the sound card/s. The parallel processing can be by cpu parallel 
processing techniques or by software control switching. Therefore both paths are always operating 
both ways to provide conversation. 

The software has to overcome the difficulty of a single sound card operating in most uses and 
generally overriding other sound cards if installed later. The software overcomes this predetermined 
intent and the unusual parallel operation of two sound cards whether that be true parallel operation or 
switching between a speed of a voice phrase of about 2 seconds to the megahertz switching of the 
cpu. 

This invention provides a practical solution to enable: 

(1) a conversation and/or dialogue (which is relatively immediate, instant and on-the-spot) 
between two persons or groups wishing to communicate by conversing in two different 
languages either face-to-face or over a telephone line (or similar); and 

(2) a speaker to communicate by addressing an audience in a language that is different to that 
of the audience. 

The invention and methodology of providing a practical solution to achieve the following is 
detailed as follows: 

• Technical methodology behind the solution in diagrams, flowcharts and by description. 

• The applications that comply with the three main scenarios of 


o Person-to-person conversation and/or dialogue in 2 different languages at any one 

instance 

o Person-to-person or party-to-party conversation and/or dialogue via a telephone 

line (or similar) in two different languages at any one instance. 

o Person to many in a lecture, conferencing, or public addressing System from one 

language to a different language at any one instance. 

that satisfies the following three specific scenarios. 

(1) Enabling a face-to-face conversation or dialogue (type method of communication) 
between speakers of two different languages. 

(2) Enabling a telephone (or similar remote) conversation or dialogue (type of 
communication) between speakers of two different languages. 

(3) Enabling a one-to-many communication between a speaker and audience in two 
different languages. 

The invention provides an innovative and practical solution to the above scenarios providing 
the ability to conununicate (speak) in Language-A and be understood (heard) in Language-B - 
immediately, instantly and "on the spot". And the ability, in reverse to communication (reply 
back) in Language-B and be understood (heard) in Language-A. As in the first two scenarios the 
ability to have a real-time conversation / dialogue in two different Languages. In the third 
scenario the ability to communicate by "addressing" or "to inform" in one language but be 
understood (heard) in a different language. 

. In order that the invention may be more readily imderstood, an embodiment will be described 
by way of illustration only with reference to the drawings wherein: 

Figure 1 is a flow chart of a real time translator in accordance with a first embodiment 
of the invention; 

Figure 2 is a diagrammatic representation of a real time translator of Figure 1 ; 

Figure 3 is a diagrammatic representation of a first use of a real time translator in 
accordance with the invention; 

Figure 4 is a diagrammatic representation of a second use of a real time translator in 
accordance with the invention; 


Figure 5 is a diagrammatic representation of a third use of a real time translator in 
accordance with the invention; 

Referring to the drawings and particularly Figure 1 there is shown in accordance with the 
invention a real time translator having a voice receiver, a voice to text converter, a text-to-text 
spoken language converter for receiving a first language and translating to a second selected 
language, a text to voice converter for converting the translated second selected language to a 
voice output and a voice emitter for emitting the voice output. 

There is parallel processing of the voice to text conversion and/or text translation and/or the text 
to voice conversion. The first and second voice receivers and first and second voice emitters are 
provided by two sound cards or two channels operating separately on a sound card. Processing 
of the voice to text conversion and/or text translation and/or the text to voice conversion is by a 
cpu or the like with software control of the sound card/s. The parallel processing can be by cpu 
parallel processing techniques or by software control switching. 

The real time translator includes two sound paths formed by two separate electronic soimd 
manipulators with associated software such that the sound of the first voice in first language 
being received can be converted to text while the translated text into the second selected 
language is being converted to voice by the second separate electronic sound manipulator with 
associated software. The separate electronic sound manipulators has two PC sound cards or the 
like, or two separate left and right channels of a single PC sound card or the like with separate 
software control. 

There is an "overlap" in the processing of conversion of first and second voice conversions to 
or from text and/or with text to text voice language translation such that the lag time between 
receiving voice and emitting translated voice is within a reasonable conversation period. Such 
period can be of the order of 2 seconds. Further to simulate conversation the voice translation 
and emission is in voice phrases substantially corresponding with voice phrasing of input voice 
such that a continual flow of spaced voice phrases simulates conversations. Generally, such 
voice phrases are a sentence or part of a sentence. 
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Still further there is an "overlap" in processing such that a first voice in a first language is 
received and translated and emitting translated voice simultaneously or apparently 
simultaneously with receiving a second voice in a second language and translating and emitting 
second translated voice. This can be by separate processing paths including the separate PC 
sound cards or the like or separate channels on a sound card or the like or by a switching system 
for switching between two processing paths at a rate to maintain reasonable real time processing 
of both paths simultaneously. 

Procedure - Step-1 . 

• When words are spoken into Microphone (1) it is made active and received as input. 

• Words spoken in Language- A is received via Microphone (1) and converted to Text. 

• Words of Language-A (in Text format) are then translated v^thin Real time translator to 
Language-B (also in Text format). 

• Real time translator then switches focus to Speaker (2) and, the Text from of the words of 
Language-B is then converted back to speech and "spoken out" through Speaker (2). 

Procedure — Step-2. 

• Words spoken in reply or any words spoken in Language-B is received via Microphone 
(2) and converted to Text. 

• Words of Language-B (in Text format) are translated within real time translator to 
Language-A (also in Text format). 

• Real time translator then switches focus to Speaker (1) and, the Text from the words of 
Language-A is then converted to speech and "spoken out" through Speaker (1). 

• All of the above happens instantly, immediately and "on-the-spot." Enabling a real-time 
conversation/dialogue between two different languages. 

The essence of the invention is to enable a conversation / dialogue between two different 
languages and as such the invention remains unchanged irrespective of the languages in which 
the conversation or dialogue is conducted in. From the beginning, conversation between the 
following languages will be the focus. These include English, Korean, French, Simplified 
Chinese, Traditional Chinese, Italian, German, Spanish, and Japanese. 

The Technical Methodology behind invention named as "Real time translator Hardware + 
Software" is as follows: 
Three (3) basic steps . 


• Receive the input-source of the spoken word and/or sentence via a channel of input (eg input 
source-one) such as a microphone or via a telephone line and convert to written text. 

• Translate the text from one language to another. 

• Speak out the translated text converted back to speech via an output channel (output 
source-two) such as a speaker from a headphone, telephone, or other. 

Step - 1 Receive Spoken Word or Sentence via an Input Source 

(1) Real time translator Software is invoked based on input from one of the two Voice Input 
Sources and will receive the input-source of the "spoken word" and/or "sentence" via a channel 
of input such as a microphone or via a telephone line, spoken by Person- 1 in Language A. 

(2) As shown in the Hardware configuration as detailed below, the invention works based on 

• TWO Sound Cards or 

• through programming, that utilises the Operating system aspects of the "left & right" 
channel capability of a single Sound Card. 

However, for the purpose of the prototype the two Sound Card, method has been used. 
Via either of these two methods, the invention of Real time translator is based on receiving 
spoken words from Voice Input devices such as. 

(1) From a microphone (of a headset or single microphone). 

(2) From a telephone line. 

(3) From a Conference or Public announcement/speaker System. 
(3) The spoken word or sentence is then converted to TEXT. 

For the prototype demonstration the software package used for this purpose was IBM's 
ViaVoice Software package, which is specifically marketed and sold by IBM for the 
development of Voice Recognition Applications such as this. However, any similar Voice 
Recognition software of which there are several on the market can be used or similar software 
can be written. But either way the overall (Real time translator) invention behind the entire 
process of Real time translator Software remains unchanged. 

Step - 2 Translate the Text. 

The input source of words/sentence that was received and converted to TEXT from Step-1 is 
then translated from one language to another. 
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Again, for the prototype demonstration the software package used for this purpose was IBM's 
Software Package of "Language Translator for Text." This software package is specifically 
marketed and sold by IBM for the development of Text translation Applications such as this. 
However, any similar Text translation software can be used of which there are several on the 
market or similar software can be vmtten. However, either way the overall Real time translator 
invention behind the entire process of Real time translator Software remains unchanged. 

Step - 3 Speak Out the Converted Text. 

The final step is - Text-to-Speech. Once Real time translator completes the TEXT 
translation, the last step is to convert back to speech and "speak out" the TEXT in words of 
translated language. 

Again, for the prototype demonstration the software package used for this purpose was the 
TTS Software Package by the Microsoft Corporation. This software package is specifically 
marketed and sold by Microsoft for the development of Text-To-Speech Applications such as 
this. However, any similar Text-To-Speech software can be used of which there are several on 
the market or similar software can be written. However, either way the overall Real time 
translator invention behind the entire process of Real time translator Software remains 
unchanged. 

Person to Person Communication via Conversation / Dialogue: 
(1) Person- 1 talks to Person-2: 

• Real time translator Hardware (Portable Hardware configured for Real time translator 
Software) - mnning Real time translator Software. Attached with Microphone/Speaker (via 
headset or other) to Sound Card-1 . 

Also attached to Sound Card-2 is another Microphone/Speaker (either fi-ee-standing or also 
via a headset) 

Sound Card-l and the corresponding Microphone & Speaker are used by Person- 1. 

Sound Card-2 and the corresponding Microphone & Speaker are for the benefit of Person-2. 

• Person- 1 speaks into Microphone attached to Sound Card-1 — those words (sentence) 
spoken in Language- A, are received by the Real time translator Software controlling input 
Microphone (1), plus the conversion to TEXT. 

• Real time translator Software controls input from Microphone (1). 
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• Real time translator Software and Software controlled by it then translates the Language- 
A text to Language-B text. 

• Real time translator Software switches control intemally within Real time translator to 
Sound Card-2, 

• The previously translated words by Real time translator of Language-B are then 
converted to speech and "spoken out-loud" and are heard by Person-2 through the Speaker 
attached to Sound Card-2. 

(2) The reverse applies when Person-2 either rephes or talks to Person- 1 : 

• Sound Card-2 and the corresponding Microphone & Speaker are for the benefit of 
Person-2. 

• Person-2 replies (or speaks) into Microphone attached to Sound Card-2 - those words 
spoken in Language-B are then received by the Real time translator Software controlling input 
from Microphone (2), plus the conversion to TEXT. 

• Real time translator Software controls input from Microphone (2). 

• Real time translator Software and Software controlled by it then translates the Language- 
B text to Language-A text. 

• Real time translator Software sv^tches control intemally within Real time translator to 
Sound Card-1, 

• The previously translated words by Real time translator of Language-A are then 
converted to speech and "spoken out-loud" and are heard by Person-2 through the Speaker 
attached to Sound Card-1. 

Thus enabling a two-way conversation between Persons 1 & 2 speaking Languages A & B 
respectively. Each would speak to the other in their respective Language and hear back from the 
other, also in their own Language. It would be almost as if there was no difference of language. It 
would be a real-time one-on-one conversation face-to-face through the portability of Real time 
translator. 

Person to Person Telephone Communication: 

(1) Person- 1 talks to Person-2 via the Telephone or similar telecommunication method: 

• Real time translator Hardware (Portable PC configured for Real time translator Software) 
- running Real time translator Software. Attached with Microphone/Speaker (via headset or 
other) to Sound Card-1 . 

Sound Card-2 is attached to the normal, industry standard Voice Modem and the output from 
the Voice Modem is then connected to a normal, standard telephone socket. No special 
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connection is required at Person-2's location and s represented by a normal telephone acting as 

another Microphone/Speaker. Therefore: 

Sound Card-1 and the corresponding Microphone & Speaker are used by Person- 1 

Sound Card-2 and the corresponding Microphone & Speaker (via telephone) are used by Person- 

2. 

• Dialling of the telephone number is done by Person- 1 using the Voice Modem and when 
a connection is made. 

• Person- 1 speaks into Microphone attached to Sound Card-l - and those words of 
Language- A is received by the Real time translator Software controlling input Microphone (1), 
plus the conversion to TEXT. 

• Real time translator Software controls input from Microphone (1). 

• Real time translator Software and Software controlled by it then translates the Language- 
A text to Language-B text. 

• Real time translator Software switches control internally vsdthin Real time translator to 
Sound Card-2. 

• The translated words of Language-B are converted to speech and "spoken out-loud" 
through the telephone line, which, is attached to the Sound Card-2 and is heard by Person-2 via 
the Speaker of the normal telephone handset. The telephone voice pulse/tone conversion is 
performed by the Voice Modem, as part of it normal ftmctionality. 

(2) Person-2 replies or talks to Person- 1 via the same Telephone or similar 
telecommunication method: 

• A reply or other words spoken by Person-2 in Language-B at the end of the Telephone line (or 
similar telecom device) is transmitted down the telephone line as normal and is input to Sound Card-2. 

• Real time translator Software controls input from Microphone (2). 

• Real time translator Software and Software controlled by it then translates the Language-B text 
to Language-A text. 

• Real time translator Software switches control internally within Real time translator to 
Sound Card- 1 , 

The translated words by Real time translator of Language-A are then switched to Sound 
Card-1, converted to speech and "spoken" and heard by Person- 1 via the Speaker (headset or 
other) attached to Sound Card-1. 

Thus enabling a two-way conversation between Persons 1 & 2 speaking Languages A & B 
respectively over a normal standard telephone line. Each would speak to the other in their 
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respective Language and hear back from the other in their own Language. It would be almost as 
if there was no difference of language. It would be a real-time one-on-one conversation face-to- 
face through the portability of Real time translator or via telephone by hooking it up to a 
telephone (as described below) 

Use of the Voice-Modem: 

The use of a normal standard Voice Modem to connect Real time translator Hardware (and 
thereby Software) is to provide a simple solution for the conversion between speech and standard 
telephone pulse/tone. It is also so that when used in different countries Voice Modems approved 
by the individual telecom authorities can be used easily and effectively instead of a specific built 
converter, which must then receive approval in each country. 

Note : 

As with the face-to-face scenario, when used over the telephone, Person-2 at the other end 
does not require Real time translator or any special device, as Real time translator on Person- 1 
performs all the work. 

Personal to Many Persons - in a speaker- to-audience or public address scenarios. 
(1) Person- 1 talks to many Persons (representing Person-2) 

• Real time translator Hardware (Portable PC configured for Real time translator Software) 
- running Real time translator Software. Attach Microphone/Speaker (via headset or stand 
alone) to Sound Card-1. 

• Attach Sound Card-2 another Microphone/Speaker (either free-standing or also via a 
headset) if audience participation required else to a loudspeaker or any other speaker/broadcast 
System. 

Sound Card-1 and the corresponding Microphone & Speaker are used by Person- 1 (the lecturer 
/speaker in the this instance. 

Sound Card-2 and the corresponding Microphone & Speaker are for the benefit of Person(s)- 
2 - the audience in this scenario. 

• Person- 1 speaks into Microphone attached to Sound Card-1 - those words of Language-A 
are received by the Real time translator Software controlling input Microphone (1), plus the 
conversion to TEXT. 

• Real time translator Software controls input from Microphone (1). 

• Real time translator Software and Software controlled by it then translates the Language- 
A text to Language-B text. 
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• Real time translator Software switches control internally within Real time translator to 
Sound Card-2, 

• The translated words by Real time translator of Language-B are then switched to Sound 
Card-2, converted to speech and "spoken out-loud" and are heard by the audience (Person-2) via 
the Loudspeaker/Speaker attached to Sound Card-2. 

Summary: 

The invention (of the Real time translator Software and Hardware) provide for an easy two- 
way conversation/ dialogue between two (2) different languages at a single instance. 

• In a face-to-face conversation (through the portability of Real time translator). 

• In a conversation conducted over a standard telephone or telecommunication. 

• In a one to many dialogue, such as a speaker to audience situation. 

• In a one to many situation such as Radio, Television broadcasts & Public announcements. 

• In a many to many dialogue, such as over a conferencing system. 

(1) Specific Hardware configuration for the Real time translator. 
The special configuration requirement was to: 

• To add two Soimd Cards. 

• The same effect can also be obtained by coding to utilise the "left & right" channel 
invention of the single Sound Card but for the prototype the two Sound Card, approach was 
taken. 

(2) Prototype 

An embodiment of the invention can be built to be portable and will be specially built to be as 
small as possible and therefore easily carried by a person. 

Real time translator software effectively breaks down the barriers of language. Whether it be 
English to Chinese or German to Japanese the difference of language and the inability to speak 
and establish a dialogue with someone unable to understand your own and only speaking a 
different language is changed forever by Real time translator. 

Real time translator is a companion and friend for the traveller and the tourist means and 
provides complete freedom. Travel freely and easily from country to country and to make 
themselves understood as well as to understand the spoken language - instantly and "on the 
spot", without requiring to study or know any language at all 

Real time translator for the businessperson provides an effective means of communication. A 
weapon and tool that provides for easy communication over the phone without the expensive and 
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wasteful exercise of wasting time and money. No language barrier & the accompanying 
problems/frustrations, talk directly to clients, supplies and potential business. 

Real time translator provides for an effective tool in mass communications, Education and 
Presentation scenarios when communication in a different language, as well as for government 
organizations requiring to deal with people speaking different languages. 

NEVILLE JAYARATNE 
By his Attorneys 
PIPERS (Melb) 

17 December, 2001 


14 


1/3 


Person-1 speaks Language-A 

using Microphone (1) 

and receives replies back in 

Language-A through Speaker 

(1) 


I 


1 


Receive words from 
Language-A and 
convert to text 


Translate text from 
Language-A to 
Language-B 


Switch to Speaker (2) 


Convert text to Speech 
and speak out-aloud 


Person-2 spealcs Language-B 

using Microphone (2) 
and receives replies back in 
Language-B through Speaker 
(2) 


1 


Receive words from 
Language-B and 
convert to text 


Translate text from 
Language-B to 

Language-A 


Switch to Speaker (1) 


Convert text to Speech 
and SDeak out-aloud 


fl^tK^e/ 1 


2/3 


Real time translator 
Hardware (PC) running 

Real time translator and 
other software 


Microphone (1) 


Speaker (1) 


Sound 

— ► 

Sound 

Card(l) 


Card (2) 





Microphone (2) 


Soeaker (2) 


Real time translator 
M 


Language - A 


Language - B 


3/3 


Headphones 






Microphone 








Speaker 



Person- 1 Real time translator (carried by Person- 1 ) Person-2 


Language A 

C1. Person to Person 


Language B 


Dl. Real time translator via the Telephone 


using microphone/speaker 
(headset) 


Person- 1 


Language A 





Voice Modem 


Language B 



Lianfifuaee B 
LoudsDeaker / Soeaker 


E1. Real time translator - One to Many or Many to Many 


